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ABSTRACT 

Background: Limited access to specialist care remains a major barrier to health care in Canada, affecting patients 
and primary care providers alike, in terms of both long wait times and inequitable availability. We developed an 
electronic consultation system, based on a secure web-based tool, as an alternative to face-to-face consultations, 
and ran a pilot study to evaluate its effectiveness and acceptability to practitioners. 

Methods: In a pilot program conducted over 15 months starting in January 2010, the e-consultation system was 
tested with primary care providers and specialists in a large health region in Eastern Ontario, Canada. We col- 
lected utilization data from the electronic system itself (including quantitative data from satisfaction surveys) and 
qualitative information from focus groups and interviews with providers. 

Results: Of 18 primary care providers in the pilot program, 13 participated in focus groups and 9 were interviewed; 
in addition, 10 of the 11 specialists in the program were interviewed. Results of our evaluation showed good uptake, 
high levels of satisfaction, improvement in the integration of referrals and consultations, and avoidance of unneces- 
sary specialist visits. A total of 77 e-consultation requests were processed from 1 Jan. 2010 to 1 Apr. 2011. Less than 
10% of the referrals required face-to-face follow-up. The most frequently noted benefits for patients (as perceived by 
providers) included improved access to specialist care and reduced wait times. Primary care providers valued the 
ability to assist with patient assessment and management by having access to a rapid response to clinical questions, 
clarifying the need for diagnostic tests or treatments, and confirming the need for a formal consultation. Special- 
ists enjoyed the improved interaction with primary care providers, as well as having some control in the decision 
on which patients should be referred. 

Interpretation: This low-cost referral system has potential for broader implementation, once payment models for 
physicians are adapted to cover e-consultation. 
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► Primary care providers manage a broad range 
of patients' health issues, with many providers requir- 
ing specialist support at some point. In Ontario alone, 
54000 patients are referred to specialists daily, the 
majority of these referrals being requested by primary 
care providers. 1 



Traditionally, when a consultation is needed, the pri- 
mary care provider sends the patient for a formal face- 
to-face consultation with the specialist. However, such 
appointments may be delayed because of lack of spe- 
cialists' availability, and the patient is frequently re- 
quired to travel some distance. 
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Waiting for specialist care remains the most import- 
ant barrier to health care access in Canada. 2 The aver- 
age wait time to see a specialist after being referred by a 
primary care provider has increased substantially over 
the past 2 decades, from 3.7 weeks in 1993 to 8.9 weeks 
in 2010. 3 After being seen by a specialist, Canadian pa- 
tients wait, on average, another 9.3 weeks to receive 
treatment. 3 The complexity of the health care system 
and the many steps involved in a specialist consultation 
also contribute to the overall length of the process (see 
Figure 1). 4 

Some primary care physicians use informal (so- 
called curbside) consultations to ask questions of 
specialists they know and meet by chance. However, 
opportunities for such informal consultations with 
specialists have been reduced, now that many primary 
care providers work completely outside the hospital set- 
ting. 5,6 In addition, concerns exist about the quality and 
adequacy of information exchanged during informal 
consultations. 6 " 9 Alternatively, a consultation may be 
done informally by telephone or e-mail. 7 ' 10,11 

The use of e-mail systems for consultation has been 
explored in a variety of settings, including military 
medical centres and clinical teaching units. 12-15 How- 
ever, confidentiality, privacy, and the security of e-mail 
systems are areas of concern 11 that have limited the po- 
tential expansion of such consultation systems. 

Telemedicine is another alternative to traditional 
consultation, especially in remote areas. 16 However, 
telemedicine is not widely used, and the require- 
ment for specialized and often expensive equipment 
makes it inaccessible to most primary care providers. 
Furthermore, studies have shown that the overall cost- 
effectiveness of telemedicine is limited. 17 In addition, it 
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Figure 1 

Steps in process of care involving both a primary care provider (PCP) and a specialist. Based on 
a prototype shared with the authors by the College of Family Physicians of Canada and a model 
proposed by the Institute for Clinical Evaluative Sciences. 4 



requires the simultaneous presence of the patient, the 
primary care provider, and the specialist. 6,18 

We were interested in the feasibility of establishing 
an electronic consultation (e-consultation) system and 
in exploring the impacts and potential benefits of such 
a system for patients and their health care providers. 
Using an existing multipurpose virtual collaboration 
space, we developed and piloted an electronic consulta- 
tion system with primary care providers and special- 
ists in Eastern Ontario. Use of a secure, region-wide 
network allowed for ease of access (similar to e-mail) 
while fulfilling all privacy requirements. Here, we re- 
port the findings of our evaluation of the pilot phase, 
which ran from 1 Jan. 2010 to 1 Apr. 2011. 

Methods 

Setting and participants. The study took place in the 
Champlain Local Health Integration Network (LHIN), 
1 of 14 regional health districts in Ontario, encom- 
passing Ottawa and its surrounding communities. The 
Champlain LHIN serves a culturally diverse region 
with a population of 1.2 million people whose chronic 
disease burdens and health outcomes are comparable 
to those of Ontario and the rest of Canada. 19 The area 
has one main urban centre with a large, multisite ter- 
tiary care hospital housing many of the specialty ser- 
vices, which provide care to people living in outlying 
rural communities. 

Participants in the pilot study consisted of 29 health 
care providers (14 family physicians, 4 nurse practi- 
tioners, and 11 specialists), all of whom were invited to 
give feedback in an interview or focus group (or both). 

We used a mixed-methods exploratory embedded 
design in which one data type supports or provides a 

secondary role in a study 
based primarily on another 
data type. The rationale 
for this approach was that 
we were seeking answers 
to multiple questions for 
which a single data type 
would have been insuffi- 
cient. 20 More specifically, 
the qualitative methods 
used in this study, which 
consisted of interviews 
and focus groups, served 
as the primary means of 
obtaining attitudinal infor- 
mation about practitioners' 
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experience with the e-consultation service. The quan- 
titative methods, based on system utilization data, 
augmented the qualitative findings and provided infor- 
mation about practitioners' actual use of the e-consul- 
tation service. 

E-consultation service. The e-consultation service was 
built upon the secure LHIN Collaboration space hosted 
at the Winchester District Memorial Hospital in Win- 
chester, Ontario. This web-based tool was developed 
and deployed for e-scheduling applications and other 
interactions among health care workers in the region. 
The e-consultation applications, including associated 
forms, automated work flows, and interface engines, 
were based on off-the-shelf components that many 
health care organizations across Canada were already 
using, an approach that we anticipated would simplify 
replication and deployment of the system in other re- 
gions. The research team developed the forms (sample 
shown in Appendix A) and work flow (diagram shown 
in Figure 2) for this project in consultation with the 
participants, without consulting any vendors. We con- 
ducted both a privacy impact assessment and a threat 
risk assessment in compliance with Ontario's Personal 
Health Information Protection Act. We also consulted 
the Canadian Medical Protective Association about 
medical risks and liability issues related to use of the e- 
consultation system. The duty of care for consultations 
requested and provided in the course of this project 
was no different than for any other consultation, such 
as a telephone consultation or a "corridor consultation." 
In fact, the e-consultation system was potentially safer 
than various informal modes of consultation, because 
the requests were automatically recorded and the spe- 
cialist had the option of requesting more information 
or declining to provide a recommendation if there was 
insufficient information. 

The e-consultation system allowed primary care 
providers to submit a patient-specific clinical question 
to a specialist, using the standardized web-based form. 
Each provider had a unique user name and password 
for logging on and could access the web forms from 
any computer with high-speed Internet access. Supple- 
mentary patient information, such as laboratory test 
results, digital images, and health history, could be in- 
cluded with the request, to assist the specialist in mak- 
ing an informed recommendation. Each consultation 
request was assigned by project staff to an appropriate 
specialist (according to the specialty required and spe- 
cialists' availability), who was given 1 week to respond. 
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Figure 2 

Work flow for e-consultation service involving primary care 
providers (PCPs) and specialists. 

Depending on the request, the specialist had the fol- 
lowing options: 

• provide answers to questions and avoid the need 
for a patient visit 
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• request additional information before providing 
advice 

• recommend a formal referral, in which case 
additional diagnostic tests or courses of treat- 
ment could be requested and started before the 
appointment 

Both the specialist and the primary care provider re- 
ceived e-mail notification at each stage of the process, 
so they knew the status of the e-consultation and could 
log back into the system to see the response and/or pro- 
vide more information as needed. In addition, a perma- 
nent record of the e-consultation was created, which 
could be downloaded into the patient's health record. 

Training. The system was easy to use and began with 
self-registration at the site. Each registration was ap- 
proved centrally once the provider's identity was 
confirmed. Training consisted of a 20-minute ses- 
sion, which could be done in person or by telephone, 
depending on the provider's level of comfort with the 
technology. Participants were given a training manual 
for future reference and a "cheat sheet," a card sum- 
marizing the steps to completing a referral (Appendix 
B). Ongoing technical and system support was provided 
by the Champlain LHIN. 

Qualitative data: focus groups and interviews. For 

focus groups and interviews involving primary care 
providers, we used purposeful sampling with max- 
imum variation, seeking to include providers from all 
sites, as described by Patton. 21 Because of the smaller 
number of specialists who participated in the pilot 
study, we conducted qualitative interviews (but no 
focus groups) with all of the specialist participants, ex- 
cept for one, who had scheduling conflicts. 

We started with 2 focus groups for all participating 
primary care providers, each lasting about 90 minutes. 
Each participant reviewed and signed a consent form 
at the beginning of the focus group and received $300 
for participation. A moderator's guide was used con- 
sistently across focus groups. Focus group discussions 
were audiotaped and transcribed verbatim. A research 
consultant (M.S.R.) then conducted more in-depth 
semi-structured telephone interviews, each about 60 
minutes in length, with primary care providers who 
met the criterion of being active users (i.e., participat- 
ing in at least one e-consultation ). In addition, all but 
one of the specialists were interviewed by the same 
research consultant. These interviews were similar, in 



terms of length and content, to the interviews with pri- 
mary care providers. All of the interviews were audio- 
taped and transcribed verbatim. 

The research consultant, an experienced qualita- 
tive researcher, developed the initial coding framework 
(which was based on the interview questions and the 
moderator's guide), coded the transcripts from the 
focus groups and interviews, and worked with other 
members of the research team to further refine the 
coding categories, which were then applied to all tran- 
scripts. Data were analyzed using NVIV08 software 
(QSR International [Americas] Inc., Burlington, Mass.), 
which identified themes by reviewing the data within 
and across codes. Coding summaries were reviewed by 
2 other members of the research team. Inconsistencies 
were resolved through discussion until consensus was 
reached within the team. Qualitative information was 
compared across participant groups and against the 
quantitative information (described below) to help in 
interpretation of the overall findings. 

Ethics approval for this study was obtained from the 
Ottawa Hospital Research Ethics Board. All primary 
care physicians and specialists provided informed con- 
sent before participating in the study. Primary care 
providers were asked to obtain each patient's verbal 
consent before using the e-consultation system for that 
patient's care. 

Quantitative data: system utilization data. For each e- 
consultation, data regarding the primary care provider, 
the consulting specialist, the clinical questions posed, 
and the answers provided were automatically stored 
by the system. Log-in time, time spent on the consul- 
tation, time for reply and closure of the case, and re- 
sponses to a satisfaction survey were all collected and 
stored permanently on the system. 

Results 

Participant characteristics. Of the 18 potential partici- 
pants in focus groups for primary care providers, 13 
agreed to participate (7 in focus group 1 and 6 in focus 
group 2). The focus group participants included both 
family physicians (n = 9) and nurse practitioners (n = 4) 
from 2 rural primary care practices, both of which were 
using electronic medical record systems. Nine primary 
care providers met the criterion of being active users 
(i.e., participated in at least one e-consultation) and 
participated in an interview. Three of these interview 
participants had also participated in a focus group. 
All 9 were using a computer or web-based device to 
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assist in their daily practice (before initiation of the e- 
consultation service). Eight of these providers prac- 
tised in a rural location, and one in a semi-rural loca- 
tion; 2 providers worked in a solo practice and 7 in a 
group practice. The median number of years in practice 
was 20 (range 2-27.5 years). The average number of 
usual referrals per week was 15. In addition, 10 special- 
ists covering 7 specialties participated in an interview, 
all of whom were using a computer or web-based device 
to assist in their daily practice. Nine of the specialists 
were in urban locations (academic practice), and 1 was 
in a rural location (community practice). The medi- 
an number of years in practice for specialists was 25 
(range 4-34 years). 

Use of e-consultation. From 1 Jan. 2010 to 1 Apr. 2011, 
77 e-consultation requests were made through the sys- 
tem by 18 primary care providers, with a response from 
1 of the 11 participating specialists. Most e-consultation 
requests were addressed to a dermatologist (19 [25%]), 
an endocrinologist (18 [23%]), a cardiologist (9 [12%]), 
or a rheumatologist (9 [12%]). Most responses (58 or 
75%) were received within the specified time frame of 1 
week, with 19 (25% of the total) being completed within 
1 day. The average response time was 5.5 days. Less 
than 10% of the e- consultations required follow-up 
face-to-face visits. In those cases, the specialist often 
asked the primary care provider to take further action 
(such as advising the patient, ordering diagnostic tests, 
or starting a course of treatment) while the patient was 
waiting for his or her consultation. 

Satisfaction. The majority of physicians and nurse 
practitioners were satisfied with the e-consultation 
service. Many physicians commented on its simplicity 
and effectiveness. One primary care provider (ID 205) 
stated, "It is pretty straightforward. I typed in my con- 
sult and sent if off." One specialist (ID 101) said, "I was 
absolutely satisfied with the type of information, the 
clarity of it, and especially the timeliness of it." 

Some specialists mentioned how useful it was to 
build their work on what another physician had already 
done. Noted one (ID 202), "It's always quicker to read 
someone's findings rather than to go ahead and do the 
full exam yourself. I probably would spend anywhere 
from 30 to 45 minutes with a new patient. What I re- 
ported as having spent on e-consultation was much less 
than that. Nothing more than 20 minutes." All special- 
ists and most primary care providers suggested that 
the e-consultation service be expanded in the future, 



and many indicated that they would recommend e- 
consultation to a colleague. 

Perceived or potential benefits. The perceived or po- 
tential benefits of e-consultation, as reported by pilot 
participants, applied to 4 main groups: patients, pri- 
mary care physicians, specialists, and the healthcare 
system as a whole (Table 1). 

For patients, the most frequently noted benefits, as 
perceived by study participants, included improved 
access to specialist care by avoidance of the need for 
face-to-face consultation or re-consultation, as well 
as reduced delays or provision of timely access. Both 
groups of providers commented on the value of the 
2-way communication facilitated by the e-consultation 
system, which allowed efficient triage of cases and 
rapid clarification of clinical questions and responses. 
Primary care providers valued the ability to assist with 
patient assessment and management by responding 
to clinical questions, clarifying the need for diagnos- 
tic tests or treatments, and confirming the need for a 
formal consultation. Specialists enjoyed receiving com- 
prehensive advance work on a case before the consul- 
tation, as this allowed them to better understand the 
patient's history and any previous investigative work 
done, as well as giving them some decision-making 
control about who should be referred. 

Areas of concern. Users of the e-consultation system com- 
mented on a few areas of concern during this pilot phase. 

The impersonal nature of communication through 
an electronic system was one concern. One specialist 
(ID 205) noted, "We are always worried about giving 
advice over the phone. And, in this case, we are giv- 
ing advice having not seen or examined the patient, 
everything being based on the information that has 
been posted [and] our working knowledge." Although 
primary care providers often seek particular specialists 
for regular referrals because of familiarity, comfort or 
an existing working relationship, this mode of selecting 
a specialist was not available to participants in the e- 
consultation pilot system. Consequently, lack of com- 
fort in using the system stemmed partly from primary 
care providers not knowing the specialist with whom 
they were consulting. Specialists also welcomed the 
prospect of making e-consultations more personal by 
having the system reveal the identity of the requesting 
primary care provider. 

Participants also commented on technical issues re- 
lated to use of the system, such as variability in typing 
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skill and in ability to learn and adopt the new technol- 
ogy. Other concerns in this area were related to privacy 
and security, use of online forms, and technical quality 
of attachments. 



Suggestions for the future. The participants provided 
suggestions for a larger-scale e-consultation service, 
including adaptation of work flow processes for refer- 
rals to account for higher volumes of e-consultation. 



Table 1 

Perceived benefits of e-consultation, as reported by participating physicians 



Theme and sub-theme 


Sample quotation 


Perceived benefits for patients 


Avoiding face-to-face consultation 


"Patients love it if 1 can just call the specialist and ask the question." (ID 104) 


Reducing delays 


"[E-consultation allowed me to] identify or clarify the urgency with which a patient should be 
seen and cut down on any other forms of communications that might take longer." (ID 219) 


Avoiding unnecessary travel 


"1 had some specific non-urgent questions so 1 sent a consult to [name of specialist] and he 
gave me some specific answers that the patient found helpful; the patient lived in [rural area] 
and was quite happy to hear from the specialist and didn't have to travel to Ottawa to get an 
endocrine consult, which takes 6 months." (FG2) 


Providing psychological reassurance 


"There may be a psychological benefit to the patient to know that their case has already been 
discussed. Because sometimes patients get very anxious especially if there's a wait involved." 
(ID 219) 


Perceived benefits for primary care providers 


Improving patient management 


"1 think that the benefit would be largely for the referring physicians in terms of patient 
management." (ID 21 9) 


Gaining confidence and comfort level 


"Almost a filtering system to reassure family doctors." (ID 202) 


Providing education and knowledge translation 


"It provides vehicles for some feedback to family docs/education to let them know how we deal 
with things so that maybe they can feel more confident dealing with things themselves." 
(ID 212) 


Improving interaction with specialists 


"And also it would make it easier if 1 get a letter back from the specialist and either 1 don't 
understand the condition they're talking about or what they've said to me seems ambiguous or 
in some way I'm not comfortable with the letter I've gotten back, it's a lot easier to [e-mail] back 
and say/What did you mean?"'(FG1) 


Perceived benefits for specialists 


Accessing primary care provider's advance work on 
a case before the consultation 


"So for me it was nice to be involved in the situation where I've got a lot more from the fam- 
ily doctor. I had a good sense of what they've tried, what they didn't try, what investigations 
they've done, everything was attached because to see it right there, you don't have to call them 
up and ask them for more." (ID 202) 


Gaining some control in decisions about which 
patients should be referred 


"[When] we get referrals to see you face to face, you book the patient in to see, you don't really 
decide necessarily that they absolutely need to see you. Whereas if you recommend it with 
e-consult, you are making the statement, you are saying that they absolutely need to see you 
because this is something that you can do." (ID 202) 


Improving interaction with primary care providers 


"I think it helps in the interaction with the health care provider. They tell you what information 
they have, you evaluate it and then if you need further information, you tell them 'This is what 
you need.'" (ID 216) 


Reducing wait times to see a specialist 


"[l]n our clinic sometimes we struggle to get in the urgent consults within a timely manner just 
because the wait times are getting longer, not just for the non-urgent but also for the urgent 
clinic appointments reducing wait times can be associated with less stress to [us] and so forth." 
(ID 211) 


Perceived benefits for health care system 


Improving efficiency 


"If things were organized in an efficient fashion so that specialists could sit down and do 10 
e-consultations over a 2-hour period, then that is better use of that specialist's time and more 
patients [are] being addressed." (ID 202) 


Reducing wait times 


"Yes 1 think the cut in wait times is a very important benefit to the health care system because 
that's a major limitation to the way that our clinics are set up. There is a long wait for patients to 
get in to see a specialist." (ID 202) 

"1 think that's where 1 could see it affecting wait times is that the consultants wouldn't be busy 
with cases that really aren't necessary." (ID 107) 


FG = focus group, ID = individual. 
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Most primary care providers suggested that the inte- 
gration of e-consultation into electronic medical re- 
cords would be a positive next step. All respondents 
envisioned a role for an "e-consultation assistant" to 
help with such tasks as triaging cases, booking ap- 
pointments when needed, scheduling e-consultations, 
and taking photographs. 

Furthermore, some form of hands-on training for 
initial set-up and access to a resource person were de- 
sired by novice users, aspects that could potentially 
be subsumed under the e-consultation assistant role. 
As one participant said, "I am wondering if I can get 
[trained on] how to implement the e-consult in my of- 
fice or if we can go 'get some training [on] how to imple- 
ment the e-consult through [electronic medical record 
software] by direct link'" (FG2). 

Interpretation 

We developed and implemented an effective e-consul- 
tation system for use within our health region. Our ap- 
proach was feasible and highly acceptable to both the 
specialists and the primary care providers who par- 
ticipated in our pilot study. The use of existing infra- 
structure and the development of a system based on 
off-the-shelf components of a commonly used soft- 
ware product means that this system could be broadly 
implemented at low cost. Although many requests for 
specialist advice and consultations for procedures will 
continue to require a face-to-face consultation, even a 
10% reduction in the number of face-to-face consulta- 
tions would be significant, translating to 5400 fewer 
consultations a day in Ontario alone. 1 Such a reduc- 
tion could reduce overall wait times for patients who 
do require face-to-face consultations and could reduce 
patients' costs (e.g., for time off work and travel). The 
estimated cost savings, even with specialists' fees for 
e-consultations factored in, could be as much as $400 
ooo/day (assuming that the cost of a usual visit is $150 
and the cost of an e-consultation is half that [i.e., $75]). 

The e-consultation system that we developed and 
implemented addressed many of the limitations and 
barriers reported with other e-consultation systems, 
such as high cost; needs for special equipment, syn- 
chronous scheduling of appointments, and extensive 
training; and privacy and security issues. 111215 ' 22 ' 23 We 
overcame these drawbacks by developing a system that 
was based on existing secure private infrastructure, 
that required less than 30 minutes of training, that was 
very convenient for providers, and that involved asyn- 
chronous communication. 



The evaluation results were encouraging, with most 
participants (both primary care providers and special- 
ists) being very satisfied with their experience of using 
the e-consultation system. Engagement and adoption 
of new technology by health care providers can be 
challenging. It has been shown that physicians who 
see innovation as offering a relative advantage over 
current practice will more readily commit to and adopt 
innovations than those who see no advantage. 24 As per- 
ceived by our participants, the use of e-consultation 
resulted in several benefits for patients, providers, and 
the health care system, including a considerable de- 
crease in the need to refer primary care patients to a 
specialist for an in-person visit. Both primary care pro- 
viders and specialists saw much value in expanding the 
e-consultation system. 

Several issues would have to be addressed in moving 
beyond the pilot stage of this project, including greater 
organization of specialty care and better integration of 
the e-consultation system into regular work flows for 
all providers. A population-based approach to deliv- 
ery of specialty services would include the creation of 
"groups" of specialists who see their mandate as provid- 
ing whole-population specialty care, as opposed to the 
current system of individual, referral-based demand. 

Looking forward, there is the question of how to in- 
corporate into this e-consultation system some of the 
best features of existing systems, such as the face-to- 
face capacity of telemedicine and the ability to deal with 
urgent questions by telephone. Furthermore, there is 
the larger issue of achieving system-wide integration, 
whereby e-consultation is linked with other health in- 
formation systems, such as regional diagnostic centres 
and laboratories, and even the possibility of using e- 
consultation as a stepping stone to an e-referral system. 

Limitations. This study was limited by its sampling bias. 
More specifically, the sampling for interviews involved 
providers who were most interested in participating in 
the e-consultation pilot. Most of the primary care pro- 
viders were from a single group practice in a rural set- 
ting, and most specialists had an academic practice in 
an urban setting. The results may not be generalizable 
to other specialist services not included in the study. 
Future studies should explore the experiences of a 
wider sample of participants. The addition of patient 
interviews would also strengthen the evaluation. 

Current status of e-consultation service. We have con- 
tinued to develop the e-consultation system, including (in 
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response to feedback from the pilot study participants) 
the creation of a self-registration system, simplification 
of the instructions for use, and automatic identification 
of the provider within the e-consultation request. At the 
time of publication, in late 2012, we were addressing 
sustainability issues related to funding mechanisms for 
e-consultation. As of 1 Nov. 2012, a total of 140 family 
doctors, 31 nurse practitioners, and 40 specialists across 
18 specialties were registered users. By the same date, a 
total of 644 consultations had been completed. 

Conclusion. This study revealed actual and potential 
benefits of e-consultation for patients, providers, and 
the health care system. By implementing this service, we 
improved access to specialty care, while ensuring that 
high-quality advice was transmitted securely, which led 
to overall cost savings for the health care system. 
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